uidelines 

for self -evaluation 



Participants in an institutional self-evaluation 
need to: 

■ make the aims, objectives and purposes of the evaluation clear to all 
members of the institution. 

■ ensure that the process is built into the structure and function of the 
institution. 

■ have a clear set of procedures for the sharing of data within and beyond the 
institution. 

■ take steps to ensure that all members of the institution believe the evaluation 
is worth doing. 

■ acknowledge that the sharing of knowledge and experience within the 
institution may be more threatening than to those outside and take steps to 
lessen this threat. 

■ treat all colleagues equally in the process of the evaluation and dissemination 
of findings. 

■ ensure that all involved in the evaluation (whether as a data givers, 
collectors or users) are engaged at some level from the start so they know 
what is happening and why. 

■ adopt methodologies that are economical and feasible to use in the time- 
scales and operations of the institution. 

■ have the backing and support of the head of the institution, including 
financial support, where appropriate for meetings, networking, dissemination 
and publication. 

■ assure members of the institution that the findings from the evaluation are fed 
back into development as well as providing a measure of accountability. 

■ indicate that the process is methodologically sound from which valid 
implications can be drawn for the precise purpose agreed. 

■ ensure the agreement and understanding of all members of the institution 
before starting the evaluation. 

■ demonstrate consistency and predictability of behaviour in the conduct and 
negotiation of the evaluation. 

■ recognise and agree when it is important to make data public and when, for 
the development of the institution, it is prudent to retain some data in 
confidence. 

■ communicate openly and honestly with colleagues, consistent with maintaining 
fair and equitable ethical procedures. 

■ seek advice and/or consider adopting a critical friend to conduct a process 
audit of the methodological rigor and fairness with which the evaluation is 
conducted. 

■ communicate to colleagues in accessible language and engage them in 
discussion on the utility of the evidence and findings. 




Guidelines for 

evaluation participants 



urn 



All participants in an evaluation shall: 

■ receive a proper explanation of the purpose and methods of the evaluation 
and should have opportunity to comment on how they are represented in the 
evaluation. 

■ receive an explanation of the evaluation agreement forming part of the 
negotiation of the evaluation teams' access to a programme. 

■ have the right to be fully informed about the purpose of the evaluation and 
procedures for collection and use of data (including explicit use of interview 
transcripts, observations and image based data). 

■ have access to the evaluation team as agreed in the contract for purposes 
of feedback, reporting and ongoing support for the duration of the evaluation. 

■ have proper opportunity to be assured that the data they offer is consonant 
with the Data Protection Act and that any data made public is on the grounds 
of fairness, accuracy and relevance. 

■ be assured that in the event of a dispute or difficulties between evaluation 
participants and evaluators, they would have access to independent arbitration. 

■ be assured that evaluators have taken all reasonable measures to ensure 
that the reports are negotiated. Final reports should normally be lodged in 
the public domain and made available to all participants. Reasons for 
exemptions need to be recorded. 
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If you have any comments about use of the guidelines in practice, please send 
them to Helen Simons h.simons@soton.ac.uk or Georgie Parry-Crooke 
georgie@parry-crooke.com 



For further copies of the guidelines please contact: 

UK Evaluation Society 
120 Wilton Road, London SW1V 1JZ 
Tel: 020 7233 8322 
Fax: 020 7233 7779 
E-mail: info@evaluation.org.uk 
Website: http://www.evaluation.org.uk 
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The UK Evaluation Society has generated these guidelines to help 
commissioners, practitioners and participants establish good practice in 
the conduct of evaluation. The guidelines attempt to capture, in an easily 
assimilated way, a diverse set of principles for action in evaluation. They 
are intended for use in any domain, discipline or context. Organised in 
different sections, the guidelines outline issues for consideration by a 
range of stakeholders involved in the evaluation process. In this way 
practice is encapsulated from the point of view of evaluators themselves, 
commissioners, participants and those involved in self-evaluation in 
organisations. 

Writing guidelines to inform practice has real challenges in that it is easy to 
slide into a set of disembodied, somewhat rationalistic standards, which 
bear little resemblance to the ebb and flow of interactions as the social 
process of evaluation unfolds. Yet it is difficult to adopt a conversational 
tone that at the same time allows for quick reference and easy access. 

These guidelines are written in a matter-of-fact style that eschews jargon, the 
obscure and the insular. They are intended for use by the novice and the 
experienced alike. While they may be a little truncated, each statement is 
designed to offer a starting point for deliberation and to act as a reference 
for statements of evaluation ethics, intentions and generic practice. In this 
sense they are educational, generated to establish a dialogue between the 
different groups on what constitutes good practice. 

The guidelines provide frameworks for action that do not exemplify any 
particular evaluation approach. Many of the statements have at their 
heart the need to be open and transparent about the expectations and 
requirements of all the stakeholders whoever they may be. As such the 
language used strives to avoid hidden assumptions about the efficacy, 
dominance or normality of any single approach to evaluation. 

The guidelines are grounded in practice, i.e., what those engaged in the 
practical business of evaluation, have found to be both honourable and 
effective ways of interacting. We believe they will come alive through use 
in discussions between people involved in evaluations and they will support 
ways of negotiating some of the critical aspects of the process from 
commissioning to dissemination of findings. 

The guidelines are not definitive and will continuously evolve; they should 
therefore be received as work-in-progress. 



Guidelines 

for commissioners 



Evaluators need to: 

■ be explicit about the purpose, methods, intended outputs and outcomes of 

the evaluation; be mindful of unanticipated effects and be responsive to shifts 
in purpose. 

■ alert commissioners to possible adjustments to the evaluation approach and 

practice; be open to dialogue throughout the process informing them of 
progress and developments. 

■ have preliminary discussion/s with commissioners prior to agreeing a 
contract. 

■ consider whether it is helpful to build into the contract forms of external 
support or arbitration (should the need arise). 

■ adhere to the terms agreed in the contract and consult with commissioners if 
there are significant changes required to the design or delivery of the evaluation. 

■ demonstrate the quality of the evaluation to other parties through progress 
reports e.g. on development and financial accountability and adhere to quality 
assurance procedures as agreed in the contract. 

■ be aware of and make every attempt to minimise any potential harmful effects 

of the evaluation prejudicing the status, position, careers or lives of participants. 

Evaluators also need to: 

■ demonstrate that the evaluation design and conduct are transparent and fit 
for purpose. 

■ demonstrate comprehensive and appropriate use of all the evidence and 

that evaluation conclusions can be traced to this evidence. 

■ work within the Data Protection Act and have procedures which ensure the 
secure storage of data. 

■ acknowledge intellectual property and the work of others. 

■ have contractual agreement over copyright of evaluation methodology, 
findings, documents and publication. 

■ write and communicate evaluation findings in accessible language. 

■ agree with commissioners from the outset about the nature of 
dissemination in order to maximise the utility of the evaluation. 

In practice evaluators need to: 

■ demonstrate a commitment to the integrity of the process of evaluation and 
its purpose to increase learning in the public domain. 

■ be realistic about what is feasible to achieve and their capacity to deliver 
within the time-scale and budget agreed. 

■ know when to refuse or terminate an evaluation contract because it is 
undoable, self-serving, or threatens to undermine the integrity of the process. 

■ be prepared to argue the case for the public right to know in evaluation in 
specified contexts. 

■ treat all parties equally in the process of the evaluation and the dissemination 
of findings. 



To ensure good practice in evaluation, it would be 
helpful if Commissioners: 

■ acknowledge the benefits of external, independent evaluation. 

■ operate fair tendering situations in which competitors ideas are not exploited 
or intellectual property misused as a result of commissioning. 

■ hold preliminary consultations with all parties to the evaluation to support a 
relevant, realistic and viable specification. 

■ specify the purpose and audience(s) for the evaluation with appropriate 
background material to encourage relevant tenders. 

■ operate a tendering procedure that is open and fair ensuring that 
appropriately qualified assessors are involved, making explicit criteria upon 
which a tender decision will be made. 

■ clarify the constraints that commissioners operate under, e.g. timescales, 
budgets, deadlines, and accountability. 

■ adhere to the terms agreed in the contract and consult with evaluators and 
other interest groups if significant changes are required to the design or 
delivery of the evaluation. 

■ specify the legal terms and responsibilities of the evaluation in the contract. 

■ match the aims and potential outcome of the evaluation to the knowledge and 
expertise of the potential evaluator(s). 

■ provide access to documentation and data required for evaluation purposes. 

■ establish clear principles for the reporting and dissemination of evaluation 
reports funded by public monies, consistent with acknowledged procedures 
which ensure quality evaluation and reporting. 

■ have realistic expectations on what an evaluation might provide including 
sufficient time for evaluators to respond to an initial invitation to tender and 
produce a proposal. 

■ include experienced evaluators (who are not potential applicants for funding) 
in initial drafts of evaluation specifications, including feasible budget and 
realistic timescales. 

■ have trust in and mutual respect for all stakeholders, participants, 
commissioners and evaluator(s). 

■ take advice of evaluators on research methodologies for collecting and 
analysing data. 

■ communicate openly and keep the evaluation team informed of changes in 
circumstances affecting the evaluation. 

■ recognise where evaluators need to keep their sources of information 
anonymous. 

■ preserve the integrity of the findings, e.g. by not quoting or publicising 
findings out of context. 



